Equivalent Dynamics from Disparate Synaptic Weights in a Prevalent Visual Circuit 
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Neural feedback-triads consisting of two feedback loops with a non-reciprocal lateral connection 
from one loop to the other are ubiquitous in the brain. We show analytically that the dynamics of this 
network topology are determined by two algebraic combinations of its five synaptic weights. Thus 
different weight settings can generate equivalent network dynamics. Exploration of network activity 
over the two-dimensional parameter space demonstrates the importance of the non-reciprocal lateral 
connection and reveals intricate behavior involving continuous transitions between qualitatively 
different activity states. In addition, we show that the response to periodic inputs is narrowly tuned 
about a resonant frequency determined by the two effective synaptic parameters. 
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Neural microcircuits are well-defined networks of neu- 
rons with connectivity patterns exquisitely adapted for 
performing specific signal-processing tasks [1, 2. 3 . One 
prevalent connectivity pattern found among neural mi- 
crocircuits consists of two feedback loops with a non- 
reciprocal lateral connection from one loop to the other 
(Fig. 1). We refer to this topology as a feedback-triad. In 
visual pathways for instance, the cortico-thalamic feed- 
back system of mammals (Fig. la) |U [5] and the isth- 
motectal feedback system of birds and reptiles (Fig. lb) 
H [9j [10] represent examples of this network topol- 
ogy- 

In this Letter, we address the fundamental question of 
how synaptic weights influence the dynamics and signal- 
processing characteristics of the feedback-triad. We show 
that the network's dynamics are not controlled by indi- 
vidual synaptic weights but rather by two algebraic com- 
binations of them that crucially depend upon the non- 
reciprocal lateral connection for the generation of com- 
plex network activity. We further show that these two 
algebraic combinations determine the resonant frequency 
of the resonance profile of the network. The insights 
gained into how synaptic weights control a network's dy- 
namics and signal-processing carry fundamental implica- 
tions for our understanding of neural development, neu- 
romodulation of network activity, and animal-to-animal 
variability of synaptic parameters. 

The feedback-triad microcircuit (Fig. lc) consists of 
three neurons with one input and five synaptic connec- 
tions with identical delays r. For simplicity, we chose a 
discrete-time recurrent neural network with a piecewise- 
linear activation function [TTJ [T21 [13] for the represen- 
tation of the triad circuitry. Specifically, the activity 
of neuron i at time t depends upon the activities Xj of 
coupled neurons j at the earlier time t — t according to 
the following equations of motion 
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FIG. 1: (Color) Neural feedback-triads, (a) In mammals, 
retinal ganglion cell (RGC) axons (black arrows) project 
to the thalamic lateral geniculate nucleus (1); this in turn 
projects to the thalamic reticular nucleus (2) and to the cor- 
tex (3). The latter two nuclei feed back to the lateral genic- 
ulate nucleus (1). (Modified after [5].) (b) In reptiles, RGC 
axons project to the optic tectum (1); this in turn projects 
to the nucleus isthmi pars parvocellularis (2) and to the nu- 
cleus isthmi pars magnocellularis (3). The latter two nuclei 
feed back to the optic tectum (1). (Modified after [5j.) Note 
the non-reciprocal lateral connection from (3) to (2) in each 
circuit, (c) The feedback-triad consists of two feedback loops 
with a non-reciprocal lateral connection between them. 



The matrix my measures the strength of connections 
between neurons i and j and the nonlinear transfer 
function a is chosen to be the max function cr(u) — 
max{0,u} u € 3?, guaranteeing nonnegative neuronal 
activities. Based on the circuit connectivity shown in 
Fig. (lc), the synaptic weight matrix is given by 
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where the feed-forward connections are denoted by latin 
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FIG. 2: (Color) Dynamics of feedback-triads. Subfigures (a-d) show converging, oscillatory, diverging and aperiodic activity 
states that result from choosing different combinations of the two effective synaptic parameters r\ and f . Subfigure (e) shows the 
color coded parameter space of activities. The white region corresponds to stable fixed points, the black region to unstable fixed 
points, the blue to orange colors represent oscillations with periods corresponding to the color and the cyan represents aperiodic 
states. Lastly, the purple region represents long period oscillations not represented on the color bar. (f) Zoom centered on the 
cross-hair indicating a region where small synaptic changes can result in different network activity, (g) Coexistence of multiple 
attractors (mulit-stability) at one combination of r\ and £ accessed by different initial conditions. 



letters and the recurrent connections by greek letters. 
Lastly, the input s is given by (si,0,0) where s\ is the 
input from the RGC axons. 

We investigate the case in which the feed-forward 
weights a and b are positive because this assignment has 
been experimentally established in the vertebrate visual 
system |14j . In addition, for simplicity we assume that 
the weight of the lateral connection c is also positive. 
Thus, the activities of neurons 2 and 3 will always be 
positive, and passage of the neuronal input through the 
nonlinear transfer function is unnecessary for these neu- 
rons. With this simplification, Eq. (1) can be reduced 
from three equations to the following 3r-cycle difference 
map for the activity of the network, taken as the activity 
of neuron 1. 

xi (t) = cr[s 1 + r]X! (t - 2r) + £ Xl (t - 3r)] . (3) 

The synaptic weights now appear only in the combina- 
tions 

rj — f3b + aa, £ = 0ac, (4) 

thereby reducing the complexity of the model from five 
synaptic parameters to two effective synaptic parameters. 

In the case of instantaneous signal transfer (i.e. r — 0) 
the network activity is stationary for all parameter val- 
ues rj and £. However, for nonzero delay the dynamical 



behavior can be complex and is qualitatively indepen- 
dent of t since Eq. (3) can be rescaled in time. Thus we 
identify the delay with a unit time step and choose the 
network input s\ = 1. By varying the effective synaptic 
parameters four types of dynamical behaviors emerge, 
namely, convergent (Fig. 2a), oscillatory (2b), divergent 
(2c), and aperiodic (2d). Color-coding these four activity 
states, a dynamical phase diagram representing network 
activity numerically in the 7j-£ phase plane has been ob- 
tained (Fig. 2e,f). This parameter space features qual- 
itatively different activity-state regions with fascinating 
geometrical boundaries. One immediate result is that 
if the synapse c is zero, the parameter £ vanishes leav- 
ing only converging and period-4 oscillations (Fig. 2e) as 
the biologically relevant activity states. Thus, the non- 
reciprocal connection between the feedback loops is cru- 
cial for generating complex network activity patterns. 

To understand the structure of the region of conver- 
gence, we first note that if the a operation is not per- 
formed in Eq. (3) the size and shape of this dynamical 
region remains unchanged. Accordingly, the linearized 
version of Eq. (3) written in standard form 

a!i(t + 3)-»ja:i(t+l)-^ai(t)-l = 0, (5) 

should yield the boundaries of the convergent region. As- 
suming a solution of the form x\{t) — A*, Eq. (5) yields 
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three eigenvalues, one real and a complex-conjugate pair. 
The stability of the fixed points in this region is guaran- 
teed when the moduli of these eigenvalues are less than 
one. By imposing this condition we determine the bound- 
aries of the stable fixed-point region to be rj 2 = £ 2 — 1 , 
£ = r\ — 1, and £ = 1 — rj. These fixed-point boundary 
results are consistent with what is found for differential- 
delay-equation models studied previously [15] . 

The structure of the limit-cycle regions in the param- 
eter space cannot be obtained from the linearized form 
of Eq. (1). Instead, they are a result of the nonlinearity 
imposed on the activities of the neurons by the trans- 
fer function. However, initial transients make it difficult 
to predict when the nonlinearity intervenes. To address 
this problem, we developed a set of constraint equations 
based on the steady-state limit-cycles obtained from our 
numerical simulation. Since the trajectory is periodic the 
activities of each neuron can be represented by a vec- 
tor whose length T is the period of the limit-cycle. The 
steady-state numerical results determine the positions of 
the zeros in each of the vectors. Elements from different 
vectors are related to each other by Eq. (1), thus giving 
3T constraint equations. When solved simultaneously, 
these equations yield polynomials in -q and £ that form 
the boundaries of the limit-cycle regions. For example, 
the period-2 region denoted by a star in Fig. (2e), has 
six constraint equations which, when solved, yield the 
boundaries £ < r\ — 1 and r] < 1. Neuromodulation of 
the synaptic weights would allow the network parame- 
ters to cross these boundaries, reconfiguring the circuit 
to produce different temporal patterns. 

On applying this procedure to higher periods, we found 
that the structure of the striations in the parameter space 
are described by geometric series in the effective synaptic 
parameters. For example, in the upper-left quadrant of 
the parameter space, the upper and lower boundaries of 
each of the striations are respectively given by 

P 2p 
i=l i=l 

and 

/ P 2p \ 

i+v E^ i_1 +E^ =°' (6b) 

\i=l i=l / 

where p, the order of the polynomial, is related to the 
period of the region between an upper and a lower stri- 
ation by the equation p = (T — l)/3. The boundary 
separating the limit-cycles from the divergent states in 
the upper-left quadrant of the parameter space is not 
analytic. Even so, we found that it is described by the 
intersection of the striation boundaries. Specifically, the 
intersection of polynomials of order p and p+1 determine 
a point along this upper left boundary of the parameter 
space. 



In addition to stable (i.e. fixed-point) activity states 
and limit-cycles, the system also exhibits aperiodic states 
(Fig. 2d). We tested these states for chaotic dynamics by 
varying the initial conditions and checking whether the 
trajectories diverged from one another. The trajectories 
are not chaotic, but they do exhibit qualitatively different 
activity depending upon the initial conditions (Fig. 2g). 
This coexistence of multiple attractors is referred to as 
multistability and has been proposed as a mechanism for 
memory storage and pattern recognition |16l 117] . 
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FIG. 3: Critical behavior near the boundary separating the 
period-2 and period-6 activity states. Plots (a,b) show the 
gathering of Fourier weights around period-6 as the path 
moves closer to the transition boundary, (c) Order param- 
eter M measured along this path shows a second-order phase 
transition, (d) Phase-space plot depicting activity responses 
for different X\ initial conditions for a point on the boundary, 
(e) Transitory time 8 measured near the critical value in the 
initial conditions. 

The parameter space has many regions in which small 
changes in the effective synaptic weights can lead to qual- 
itatively different network activity states. To understand 
the transition from one activity state to another, we 
employed an order parameter M involving the Fourier 
transform of the network activity. This was motivated 
by the observation that the Fourier transform of the ac- 
tivity near a regional boundary contained Fourier com- 
ponents consistent with periods corresponding to that 
region and the adjacent region (Fig. 3a,b). The order 
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parameter M = 1 — Ylk'ytk m x /Sfc compares the 
power Ik of each peak k in the Fourier transform relative 
to the total power of the spectrum. As the boundary 
separating the period-2 (star in Fig. 2e) and period-6 ac- 
tivity states is crossed, a second-order phase transition 
occurs (Fig. 3c), indicating that points along the bound- 
aries separating different network states will be sensitive 
to the initial network activity. To quantify this critical 
behavior, we determined numerically the network activ- 
ity for various initial conditions (Fig. 3d) and measured 
the length of the transitory time 9 before the network 
settles into steady-state activity (Fig. 3e). The results 
show that the boundaries are multistable, a feature that 
allows the circuit to show activity-dependent responses, 
such as are observed experimentally in the mammalian 
visual system |18j . In addition, the boundaries of various 
regions exhibit a devil's staircase, (Fig. 3e), a self-similar 
fractal structure with scaling indexes universal to a large 
class of dynamical systems [19] . 

An important signal-processing characteristic is the 
susceptibility of the feedback-triad to periodic inputs 
with specific driving frequencies to. In Fourier space, 
the linear response r(uS) of the triad is related to the 
stimulus s{uj) by r(u>) = x(lu)s(u>), where x(w) is the 
AC susceptibility. This relationship holds for sufficiently 
weak driving input. For example, at the point (77, £) = 
( — 1.5,-1.5), the neural susceptibility profile was found 
to follow a scaled Lorentzian of the form 
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where the (narrow) Lorentzian width T = 2.5 x 10~ 4 , the 
scale factor b = .72725, and the Lorentzian neural reso- 
nant frequency ujq = 0.2 (Fig. 4). Throughout this work, 
we set the time step r to unity. For a general time step 
r, the values of both a and ujq quoted above are in units 



of- 



This sharp tuning provides an effective bandpass 



filter that enables the extraction of a specific frequency 
from the input stimulus. Furthermore, since the resonant 
frequency depends on the values of the synaptic weights, 
neuromodulatory substances would allow the microcir- 
cuit to switch modes of preferred stimulus frequency. A 
similar analysis could be extended to larger networks 
comprised of feedback-triad microcircuits in "series" . For 
the case of N feedback-triads the response would be re- 
lated to the stimulus by r(u) = (j]^ Xi(u)^J s(lu) where 

Xi(^) is the susceptibility of the i th feedback-triad. 

Numerous neural and biochemical [ST] networks 
can be mapped onto the feedback-triad network topol- 
ogy we have investigated. Importantly, it has been shown 
that the dynamics and signal processing characteristics of 
this topology are determined by two combinations of the 
synaptic weights that depend upon the lateral connection 
for the generation of complex dynamics. The implica- 
tions of this study are fourfold. First, the seeming incon- 
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FIG. 4: Susceptibility profile of the feedback-triad over fre- 
quencies given in units of r . The susceptibility is sharply 
tuned about a resonant frequency determined by the two ef- 
fective synaptic weights. See text and Eq. tffj. 



gruity in animal-to-animal variability of synaptic weights 
and equivalent dynamics is addressed because network 
activity is not determined by individual synapses but 
rather by certain combinations of the synaptic weights. 
Second, manipulation of individual synapses by biochem- 
ical agents must likewise be envisioned in terms of com- 
binations of synaptic weights. Third, the existence of 
continuous transitions between different activity states 
in the two-dimensional parameter space highlights the 
flexibility of the dynamics in terms neuromodulation of 
the synaptic weights and activity-dependent responses. 
Last, the signal response is sharply tuned around a nat- 
ural resonant frequency determined by the two effective 
synaptic weights, indicating that larger networks com- 
posed of feedback-triad microcircuits may be suited to 
bandpass filtering of neural stimuli. 
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